An Application of Importance-based Feature Extraction in Reinforcement Learning

|The sparse feedback in reinforcement learning problems makes feature extraction diicult. We present importance-based feature extraction, which guides a bottom-up self-organization of feature detectors according to top-down information as to the importance of the features; we deene importance in terms of the reinforcement values expected as a result of taking diierent actions when a feature is recognized. We illustrate these ideas in terms of the pole-balancing task and a learning system which combines bottom-up tuning with a distributed version of Q-learning; adding importance-based feature extraction to the detector tuning resulted in faster learning.